Machine data can be utilized to document the activity and behavior for information technology (“IT”) systems. IT systems are constantly logging their own machine data, and, therefore generate a vast amount and wide variety of machine data such as IT activity logs, system or device configuration data, network messages, database records, application programming interfaces (“APIs”) data, telephone records, sensor data, etc.
Activities such as troubleshooting IT systems, detecting operational trends, catching security issues, and measuring business performance, generally require organizing and understanding machine data. System administrators have a need to access and comprehend the machine data from one or more components of an IT system for the purpose of locating and correcting problems during operation, security analysts have a need to understand patterns of machine data behavior to identify potential security threats, and business people have a need to trace machine data across various components of an IT system to follow the activities customers perform when purchasing products or services. But the overwhelming volume, different and changing formats, and overall complexity of machine data create significant difficulty for software developers, system administrators, and business people who need to gain insight into IT system functions and behaviors.
This problem is compounded by the fact that information systems—and the machine data they generate—continue to grow in complexity and size. Machine data comes in a wide array of unpredictable formats and conventional monitoring and analysis tools are not designed for the variety, velocity, volume, or variability of this data. In addition, conventional methods of organizing and understanding machine data across multiple information systems and domains suffer from an inability to effectively keep up with changing machine data.